Multimedia containing various content types including text, audio and video, provides an outstanding business and revenue opportunity for network operators. The availability of higher bandwidth and the use of packet-switched Internet Protocol (IP) technology have made it possible to transmit richer content that include various combinations of text, voice, still and animated graphics, photos, video clips, and music. In order to capitalize on this market potential network operators must meet customers' expectations regarding quality and reliability. Transcoding of media at server level is crucial for rendering multimedia applications in today's heterogeneous networks composed of mobile terminals, cell phones, computers and other electronic devices. The adaptation and transcoding of media must be performed at the service provider level because individual devices are often resource constrained and are rarely capable of adapting the media themselves. This is an important problem for service providers, as they will have to face a very steep traffic growth in the next few years; growth that far exceeds the speed up one can obtain from new hardware alone.
Multimedia Messaging Services, for example, provide server-side adaptation to ensure interoperability between terminals. Adaptations required for multimedia messaging are discussed by S. Coulombe and G. Grassel, in “Multimedia adaptation for the multimedia messaging service,” published in IEEE Communications Magazine, volume 42, number 7, pages 120-126, July 2004. The most frequent image-related interoperability issues do not involve image formats, as the majority of the traffic involves JPEG and GIF images, but rather a resolution or file size exceeding the capabilities of the receiving terminal. For instance, the limited memory of some mobile phones requires images to be under a certain size or resolution in order to be received and displayed. Moreover, the height and the width of the image should be such that it can be accommodated by the terminal with a given set of characteristics. Changing an image's resolution, or scaling, to meet a terminal's capabilities is a problem with well-known solutions. However, optimizing image quality against terminal constraints remains a challenge, as there are no well-established relationships between the quality factor (QF) used in JPEG (or the number of colors used in GIF), perceived quality, and the compressed file size. Using scaling as an additional means of achieving file size reduction, rather than merely resolution adaptation, makes the problem all the more challenging. Several studies have investigated the problem of file size (or bit rate) reduction for visual content. Examples include the paper by Z. Lei and N. D. Georganas, entitled “Accurate bit allocation and rate control for DCT domain video transcoding,” in Proceedings of the IEEE CCECE 2002. Canadian Conference on Electrical and Computer Engineering, 2002, volume 2, pages 968-973, the paper by J. Ridge entitled “Efficient transform-domain size and resolution reduction of images,” published in Signal Processing: Image Communication, volume 18, number 8, pages 621-639, September 2003 and the US patent by V. Ratnakar and V. Ivashin, entitled “File size bounded JPEG transcoder,” May 2001, U.S. Pat. No. 6,233,359. The results of these studies show that reduction can be achieved through adaptation of quantization parameters, rather than through scaling. For most studies, this makes sense, since they were carried out in the context of low bit rate video, where resolution is often limited to a number of predefined formats. However, even in the context of still-picture coding, scaling as an adaptation strategy is not considered. For instance, Ridge provides excellent methods for scaling and then reducing the file size of JPEG images but does not consider estimating scaling and quality reduction in combination. This seems to be a major shortcoming, because the best strategy for maximizing user experience may well be to scale down the picture and compress it with a higher QF, rather than simply re-compressing it with a lower QF. Applicants Steven Pigeon and Stephane Coulombe have presented methods to estimate the compressed file size of a JPEG image subject to scaling and QF changes in “Computationally efficient algorithms for predicting the file size of JPEG images subject to changes of quality factor and scaling” published in Proceedings of the 24th Queen's Biennial Symposium on Communications, Queen's University, Kingston, Canada, 2008. It is noted that several combinations of QF and scaling lead to the same approximate file size, raising the question of which combination would maximize user experience, or image quality.
Before discussing how the present invention addresses the issues discussed earlier, a brief description of a typical prior art transcoding environment is presented. JPEG is a popular technique for compressing images contained in MMS messages. The JPEG standard provides a commonly used method for image compression. As is well known, JPEG compression is “lossy”, that is a compressed image may not contain 100% of the digital information contained in the original image. The loss of information can be controlled by setting the quality factor during the compression. A lower quality factor is equivalent to higher compression and generally leads to a smaller image file size. Conversely, a higher quality factor leads to a larger image file size, and generally higher perceived “quality” of the image.
By way of example, FIG. 1 illustrates a multi-media messaging (MMS) system architecture 100, including an Originating Node 102, a Service Delivery Platform 104, a Destination Node 106, and an Adaptation Engine 108. The Originating Node 102 is able to communicate with the Service Delivery Platform 104 over a Network “A” 110. Similarly the Destination Node 106 is able to communicate with the Service Delivery Platform 104 over a Network “B” 112. The Networks “A” and “B” are merely examples, shown to illustrate a possible set of connectivities, and many other configurations are also possible. For example, the Originating and Destination Nodes (102 and 106) may be able to communicate with the Service Delivery Platform 104 over a single network; the Originating Node 102 may be directly connected to the Service Delivery Platform 104 without an intervening network, etc.
The Adaptation Engine 108 may be directly connected with the Service Delivery Platform 104 over a link 114 as shown in FIG. 1, or alternatively may be connected to it through a network, or may be embedded in the Service Delivery Platform 104.
In a simple case, the Originating Node 102 may send a (multimedia) message that is destined for the Destination Node 106. The message is forwarded through Network “A” 110 to the Service Delivery Platform 104 from which the message is sent to the Destination Node 106 via Network “B” 112. The Originating and Destination Nodes (102 and 106) may for instance be wireless devices, the Networks “A” and “B” (110 and 112) may in this case be wireless networks, and the Service Delivery Platform 104 may provide the multimedia message forwarding service.
In another instance, the Originating Node 102 may be a server of a content provider, connected to the Service Delivery Platform 104 through a data network. Thus, Network “A” 110 may be the internet, while Network “B” 112 may be a wireless network serving the Destination Node 106 which may be a wireless device.
An overview of server-side adaptation for the Multimedia Messaging Service (MMS) is given in a paper “Multimedia Adaptation for the Multimedia Messaging Service” by Stephane Coulombe and Guido Grassel, IEEE Communications Magazine, volume 42, number 7, pages 120-126, July 2004.
In the case of images in particular, the message sent by the Originating Node 102 may include an image, specifically a JPEG encoded image. The capabilities of the Destination Node 106 may not include the ability to display the image in its original form, for example because the height or width of the image in terms of the number of pixels, that is the resolution of the image, exceeds the size or resolution of the display device or terminal in the Destination Node 106. In order for the Destination Node 106 to receive and display it, the image may be modified in an Image Transcoder 116 in the Adaptation Engine 108 before being delivered to the Destination Node 106. The modification of the image by the Image Transcoder 116 typically may include scaling, i.e. change the image resolution, and compression.
Thus, there is a need in the industry for an improved method and system for transcoding images that address the limitations of the prior art discussed earlier and take image quality and speed of transcoding into account.